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ABSTRACT 

We have developed an application called Wikipedia Live 
Monitor that monitors article edits on <lifferent language 
versions of Wikipedia — as they happen in realtime. Wikipedia 
articles in different languages are highly interlinked. For ex- 
ample, the English article "en:2013_Russian_meteor_event" 
on the topic of the February 15 meteoroid that exploded 
over the region of Chelyabinsk Oblast, Russia, is interlinked 
with "ru:nafleHHe_MCTeopHTa Ha_ypa.nc B 2013 rofly", 
the Russian article on the same topic. As we monitor mul- 
tiple language versions of Wikipedia in parallel, we can ex- 
ploit this fact to detect concurrent edit spikes of Wikipedia 
articles covering the same topics, both in only one, and in 
different languages. We treat such concurrent edit spikes 
as signals for potential breaking news events, whose plau- 
sibility we then check with full-text cross-language searches 
on multiple social networks. Unlike the reverse approach 
of monitoring social networks first, and potentially checking 
plausibility on Wikipedia second, the approach proposed in 
this paper has the advantage of being less prone to false- 
positive alerts, while being equally sensitive to true-positive 
events, however, at only a fraction of tfie processing cost. 
A live demo of our application is available online at the 
URL http://wikipedia-irc.herokuapp.com/, the source 
code is available under the terms of the Apache 2.0 license 
at https : //github . com/tomayac/wikipedia-irc. 
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1. INTRODUCTION 

1.1 Motivation 

Shortly after the celebrity news website TMZ broke the 
premature news that the King of Pop Michael Jackson (MJ) 
had died,^ the Internet slowed down.^ Initially, Wikipedia's 
website administrators started noting abnormal load spikes [8] . 
Shortly afterwards, caching issues caused by a so-called edit 
war [1] led the site to go down: Wikipedia editors worldwide 
made concurrent edits to the Michael Jackson Wikipedia ar- 
ticle, doing and undoing changes regarding the tense of the 
article, death date, and the circumstances of the (at the 
time) officially still unconfirmed fatality. While Wikipedia 
engineers have worked hard to ensure that future load spikes 
do not take the site down again, there is without dispute 
a lot of research potential in analyzing such editing activity. 

1.2 Hypotheses and Research Questions 

In this paper, we present an application that monitors 
article edits of different language versions of Wikipedia in 
realtime in order to detect concurrent edit spikes that may 
be the source of breaking news events. When a concurrent 
edit spike has been detected, we use cross-language full-text 
searches on social networks as plausibility checks to filter out 
false-positive alerts. We are led by the following hypotheses. 

(HI) Breaking news events spread over social networks, in- 
dependent from where the news broke initially. 

(H2) If a breaking news event is important, it will be re- 
flected on at least one language edition of Wikipedia. 

(H3) The time between when the news broke first and the 
news being reflected on Wikipedia is considerably short. 

These hypotheses lead us to the research questions below. 

(Ql) Can concurrent Wikipedia edit spikes combined with 

social network plausibility checks capture major break- 
ing news events, and if so, with what delay? 

(Q2) Is the approach Wikipedia first, social networks second 
at least as powerful as the reverse approach? 

In this paper, we do not answer all research questions yet, 
however, lay the foundation stone for future research in this 
area by introducing the Wikipedia Live Monitor application. 

'^MJ dead: http://www.tmz.com/2009/06/25/michael-jackson- 
dies- death- dead- cardiac- arrest/, accessed 02/18/2013 
^Internet slow-down: http://news.bbc.co.Uk/2/hi/technology/ 
8120324.stm, accessed 02/18/2013 



2. RELATED WORK 

We refer to an event as breaking news, if the event is of 
significant importance to a considerable amount of the pop- 
ulation. Petrovic et al. define [5] the goal of new event detec- 
tion (or first story detection) as "given a sequence of stories, 
to identify the first story to discuss a particular event." They 
define an event as "something that happens at some specific 
time and place." Classic streaming analysis of social network 
microposts so far has been mainly focused on Twitter, a mi- 
croblogging social network that provides access to a sampled 
stream of generated microposts by means of its Streaming 
API.'^ Petrovic ct al. explain [5]: "in the streaming model 
of computation, items arrive continuously in a chronological 
order, and have to be processed in bounded space and time." 
In the referenced paper, the authors report on a system for 
streaming new event detection applied to Twitter based on 
locality sensitive hashing. Hu et al. provide an analysis 
of how news break and spread on Twitter [3]. The task of 
linking news events with social media is covered by Tsagkias 
et al. in [7]. With this paper, we stand on the shoulders* 
of Osborne et al. [4] , who use Wikipedia page view statis- 
tics® as a means to filter spurious events stemming from 
event detection over social network streams. Our approach 
reverses theirs, however, instead of the only hourly updated 
page view statistics, we use realtime change notifications, as 
explained in Subsection 3.1. Our Wikipedia Live Monitor is 
partly based on an application called Wikistream, developed 
by Ed Summers et al., which was described in [6]. 

3. IMPLEMENTATION DETAILS 
3.1 Wikipedia Recent Changes 

As described earlier, our application monitors concurrent 
edit spikes on different language versions of Wikipedia. In 
the current implementation, we monitor 42 different Wiki- 
pedias, 5 with > 1,000,000 and 37 with > 100,000 arti- 
cles.^ Changes to any single one article are communicated 
by a chat bot over Wikipedia's own Internet Relay Chat (IRC) 
server (irc.wikimedia.org),'' so that parties interested in the 
data can listen to the changes as they happen. For each 
language version, there is a specific chat room following the 
pattern "#" + language + ".wikipedia". For example, changes 
to Russian Wikipedia articles will be streamed to the room 
#ru.wikipedia. A special case is the room #wikidata. wikipedia 
for Wikidata [9] , a platform for the collaborative acquisition 
and maintenance of structured data to be used by Wikime- 
dia projects like Wikipedia. A sample chat message with the 
components separated by the asterisk character '*' announc- 
ing a change can be seen in the following. "[[Juniata River]] 
http://en.wikipedia.org/w/index.plip?diff=516269072&oldid=514- 
659029 * Johanna-Hypatia * (-1-67) Category:Place names of Na- 
tive American origin in Pennsylvania". The message compo- 
nents are (i) article name, (ii) revision URL, (iii) Wikipedia 
editor handle, and (iv) change size and change description. 

^Twitter Streaming API: littps://dev.twitter.com/docs/api/l. 

1/get/statuses/sample, accessed 02/18/2013 
^Hence the title of this paper. 

®Page view statistics for Wikimedia projects: http://dumps. 
wikimedia.org/other/pagecounts-raw/, accessed 02/18/2013 
®List of Wikipedias by size: http://meta.wikimedia.org/wiki/ 
List_of_Wikipedias, accessed 02/18/2013 
'^Raw IRC feeds of recent changes: http://meta. wikimedia. 
org/wiki/IRC/Channels#Raw_feeds, accessed 02/18/2013 



3.2 Article Clusters 

We cluster edits of articles about the same topic, but writ- 
ten in different languages, in article clusters. The example 

of the English "en:2013_R,ussian_meteor_evcnt" and the Rus- 
sian article "ru:nafleHHe_MeTeopHTa_Ha_ypajie_B_2013_rofly" 
that are both in the same cluster illustrate this. We use the 
Wikipedia API to retrieve language links for a given article. The 
URL pattern for the API is as follows, http: //$LANGUAGE . 
wikipedia . org/w/api .php?action=query&format=json&- 
prop=langlinks&titles=$ARTICLE. 

3.3 Comparing Article Revisions 

The Wikipedia API provides means to retrieve the actual changes 
that were made during an edit including additions, deletions, 
and modifications in a dif f-like manner. The URL pattern is 
as follows, http : //$LANGUAGE . wikipedia . org/w/api . php? 
act ion=compare&torev=$TO & f romrev=$FROM& f ormat = j son. 
This allows us to classify edits in categories, like, e.g., negligible 
trivial edits (punctuation correction) and major important ed- 
its (new paragraph for an article), which helps us to disregard 
seemingly concurrent edits in order to avoid false-positive alerts. 

3.4 Breaking News Criteria 

Our application Wikipedia Live Monitor puts detected article 
clusters in a monitoring loop in which they remain until their 
time-to-live (240 seconds) is over. In order for an article cluster in 

the monitoring loop to be identified as breaking news candidate, 
the following breaking news criteria have to be fulfilled. 

> 5 Occurrences: An article cluster must have occurred in at 

least 5 edits. 

< 60 Seconds Between Edits: An article cluster may have at max- 

imum 60 seconds in between edits. 

> 2 Concurrent Editors: An article cluster must have been edited 

by at least 2 concurrent editors. 

< 240 Seconds Since Last Edit: An article cluster's last edit may 

not be longer ago than 240 seconds. 

The exact parameters of the breaking news criteria above were 
determined empirically by analyzing Wikipedia edits over sev- 
eral hours and repeatedly adjusting the settings until major news 
events happening at the same time were detected. The resulting 
dataset split into three chunks has been made publicly available.* 

3.5 Social Network Plausibility Checks 

When a breaking news candidate has been identified, we use 
cross-language full-text social network searches on the social net- 
works Twitter, Facebook, and Google+ as a plausibility check. As 
the article titles of all language versions of the particular article's 
cluster are know, we use these very article titles as search queries 
for cross-language searches, eis can be seen in Figure 1. This ap- 
proach greatly improves the recall of the social network search, 
however, requires either automatic translation, or an at least basic 
understanding of the languages being searched in. Currently the 
plausibility checking step is not yet fully automated, as the search 
results are for the time being meant to be consumed by human 
evaluators. Driven by (HI), we assume breaking news events are 

^Wikipedia Live Monitor dataset: https://www.dropbox. 
com/sh/2qsglzhb8p35fxf/Dghn55y0kh, accessed 02/18/2013 



M riaAeHHe MereopHTa HaVpane b 2013 rofly (just now) 

Versions: ■ 2013 Russian meteor event, El Meteorito de Cheliabinsk, s Meteorit de Txeli^binsk 
Conditions: £5 Occurrences V £ 60 Seconds Between Edits ^ s £ Concun^nt Editors 
Last Edit Intervals: / seconds, seconds, 30 seconds, 21 seconds 
Occurrences: 12 

Editors (9): AlphaRiio (ru), Harryziiber (en), Pdeilani (pt), Acelll (ru), Cataiaalatac (ca, en), PoBepr P3ii (ru), 1BS.1 74.1 52.1 61 (en), Medeis(en), 213.11 0.4.227 (ru) 
Languages (4): ru (6), en (4), pt (1), ca (1) 

Last Changes: 1 nninute ago l^o5epT Psii (ru, 44), just now Cataiaaiatac (ca, +5Q), just now Medeis (en, -10), just now 213.1 1 0.4.227 (ru, +229), just now Po6epT Psii (ru, 

+48) 

Social Nefworli Opinions: 

2013 Russian meteor event 

^ Jim Carr f11 minutes aoa ) Probably a lot o1 meteorite bunters Ijooking flights right ry>w. 3013 Russian meteor event - Wiltipedia. the free encyclopedia http://goo.gi/3wp1B 

Q Prepper Chicks fSO minutes aoo ^: Some history lor you. .. h1tp://www.foxrtews.com/science/2013^02^15/rLssiarvmeteor-blas1-r6calls-massive-10B-tLrtgLska-event/ Russian meteor biast [ 
^ Q Paui Riley f47 minutes aqo ^: Will anyone be stayirwj up tonight as we rtervoLsly await the fly-by of asteroid DAM? [drumrollj I dortt krtow wfiat to do in all bor^esty, 1 mean [.,.] 
Meteorito de Cheliabinsk 

^ 3^ @YavoMkt f55 minutes aoo l: "@PresidBrM:ia_EPN: Seftala rfYoSoy133 que el meteorito de Cheliabirtsk. Rusia. es una cortina de humo dei PRI para... bt1p://t.co/ir^&KlMd 
^ ^ @eril event ana fSD minutes aoo l: Ya saiio ei mere ado de los meteorites caidos: bt1p://t.co/d4UDz3jD 



Figure 1: Screenshot with an article cluster of four concurrently edited articles (ru, en, pt, ca). All breaking news criteria are 
fulfilled, the cluster is a breaking news candidate. Cross-language social network search results for en and pt can be seen. 



being discussed on social networks. We will show arguments for 
this assumption in Section 4. For now, we expect social networks 
to be a short period ahead of Wikipedia. In consequence, if the 
human rater can find positive evidence for a connection between 
social network activities and Wikipedia edit actions, the breaking 
news candidate is confirmed to indeed be breaking news. 

3.6 Application Pseudocode 

The Wikipedia Live Monitor appHcation has been implemented 
in Node.js, a server side JavaScript software system designed for 
writing scalable Internet applications. Programs are created using 
event-driven, asynchronous input/output operations to minimize 
overhead and maximize scalability. Listing 1 shows the pseu- 
docode of the two main event loops of the Wikipedia Live Mon- 
itor application. The actual implementation is based on Martyn 
Smith's Node.js IRC library^ and the WebSockets API and pro- 
tocol [2], wrapped by Guillermo Rauch's library Socket. lO.^" 

4. PREMATURE EVALUATION 

As noted earlier, in this paper, we do not answer all research 
questions yet. Nevertheless, we extract trends from our experi- 
ments so far. In Subsection 1.2, we have set up three hypotheses. 
(HI) has been proven by Hu et al. in [3] for Twitter. We argue 
that it can be generalized to other social networks and invite the 
reader to have a look at our dataset, where the lively discussions 
about breaking news candidates on the considered social networks 
Twitter, Facebook, and Google+ support the argumentation. It 
is hard to prove (H2), as the concept of important breaking news 
is vague and dependent on one's personal background, however, 
all evidence suggests that (H2) indeed holds true, as, to the best 
of our knowledge and given our background, what the authors 
consider important breaking news is represented on at least one 
language version of Wikipedia. (H3) has been examined by Os- 
borne et al. in [4]. In the paper, they suggest that Wikipedia lags 
about two hours behind Twitter. It has to be noted that they look 
at hourly accumulated page (article) view logs, where we look at 
realtime article edit log streams. Our experiments suggest that 
the lag time of two hours proposed by Osborne et al. may be too 

^Node.js IRC library: https://github.com/martynsmith/node- 
irc, accessed 02/18/2013 

^"Socket. 10 library: http://socket.io/, accessed 02/18/2013 



conservative. An educated guesstimation at this stage is that the 
lag time for breaking news is more in the range of 30 minutes, 
and for global breaking news like celebrity deaths in the range of 
five minutes and less, albeit the edits by our experience will be 
small and iterative {e.g., "X is a" to "X was a", or the addition 
of a death date), followed by more consistent thorough edits. 

The (at time of writing) recent breaking news event of the res- 
ignation of Pope Benedict XVI helps respond to (Ql). The three 
first edit times after the news broke on February 11, 2013 of the 
Pope's English Wikipedia article^'^ are as follows (all times in 
UTC): 10:58, 10:59, 11:02. The edit times of the French article^^ 
are as follows: 11:00, 11:00, 11:01. This implies that by looking at 
only two language versions (the actual number of monitored ver- 
sions is 42) of the Pope article, the system would have reported 
the news at 11:01. The official Twitter account of Reuters an- 
nounced^^ the news at 10:59. Vatican Radio's announcement'^* 
was made at 10:57:47. 

Not all breaking news events have the same global impact as 
the Pope's resignation, however, the proposed system was shown 
to work very reliably also for smaller events of more regional im- 
pact, for example, when Indian singer Varsha Bhosle commit- 
ted suicide'^ on October 8, 2012. A systematic evaluation of 
(Ql) compulsorily can only be done by random samples, which 
has turned out positive results so far. Again, we invite the reader 
to explore our dataset and to conduct own experiments. A sys- 
tematic evaluation of (Q2) requires a commonly shared dataset, 
which we have provided, however, at this point in time, we do 
not have access to the system of Osborne et al. 

Regarding Wikipedia Live Monitor's scalability, we could scale 
the monitoring system up to currently all 284 Wikipedias on 
a standard consumer laptop (mid-2010 MacBook Pro, 2.66 GHz 
Intel Core 2, 8 GB RAM), which once more proves the efficiency of 



^^Edit history en: http://en.wikipedia.org/w/index.php?title= 

Pope_Benedict_XVI&action=history, accessed 02/18/2013 
^^Edit history fr: http://fr.wikipedia.org/w/index.php?title= 

Beno%C3%AEt_XVI&action=history, accessed 02/18/2013 
^^Reuters announces Pope resignation: https://twitter.com/ 

Reuters/status/300922108811284480, accessed 02/18/2013 
^''Vatican Radio announces Pope resignation: http://de. 

radiovaticana.va/Articolo.asp?c=663810, accessed 02/18/2(313 
Varsha Bhosle suicide: http://en.wikipedia.org/wiki/Varsha_ 

Bhosle, accessed 02/18/2013 



the Nodc.js architecture for this kind of event-driven applications. 
In practice, however, the majority of the smaller Wikipedias be- 
ing very rarely updated, we limit ourselves to the Wikipedias with 
> 100,000 articles at no remarkable loss of recall. 

5. FUTURE WORK 

Future work towards a thorough scientific evaluation will mainly 
address two areas. First, the automatic categorization of edits on 
Wikipedia needs to be more fine-grained. In the context of break- 
ing news detection, not all edits are equally useful. An image 



Input: ire, listening on Wikipedia recent changes 

Output; breakingNewsCandidates, breaking news candidates 

monitoringLoop = {} 
articleClusters = {} 
breakingNewsCandidates = {} 

# Event loop 1: 

# When a new message arrives 
ire . on . message do (article) 

langRefs = get LanguageRef erences ( a r t i c 1 e ) 
articleRevs = get Ar t i c le Re vis ions ( a r t i c 1 e ) 
cluster = cluster Articles ( article , langRefs) 

# Create new cluster for previously unseen article 
if cluster not in monitoringLoop 

monitoringLoop . push( cluster ) 

articleClusters . push ( cluster ) 

updateStatistics(cluster) 

emit . newCluster ( cluster , articleRevs) 

# Update existing cluster, as the article was seen before 
else 

updateStatistics( cluster) 

emit . existingCluster ( cluster , articleRevs) 
# Check breaking news criteria 
if cluster . occurrences >= 5 

if cluster . secsBetweenEdits <= 60 
if cluster . numEditors >= 2 

if cluster. secsSinceLastEdit <= 240 
socialNetworks . search ( langRefs ) 
breakingNewsCandidates. push(cluster) 
emit. breakingNewsCandidate( cluster) 
end if 
end if 
end if 
end if 
end if 

return breakingNewsCandidates 
end do 

# Event loop 2: 

# Remove too old clusters regularly 
timeout . every .240 seconds do 

for eax;h cluster in monitoringLoop 

if cluster . secsSinceLastEdit >= 240 
monitoringLoop . remove{ cluster ) 
articleClusters . remove (cluster) 
end if 
end for 
end do 



Listing 1: The two main event loops of the appHcation 



being added to an article is an example of an edit that usually 
will not be important. In contrast, the category "Living people" 
being removed from an article is a strong indicator of breaking 
(sad) news. Second, the connection between social network search 
and Wikipedia edits needs to be made clearer. In an initial step, 
the concrete changes to an article, as detailed in Subsection 3.3, 
can be compared with social network microposts using a cosine 
similarity measure. More advanced steps can exploit the potential 
knowledge from Wikipedia edits {e.g., category "Living people" 
removed implies a fatality). 

6. CONCLUSIONS 

In this paper, we have shown an application called Wikipedia 
Live Monitor and released its source code under the Apache 2.0 
license. This application monitors article edits on 42 diflferent 
language versions of Wikipedia. It detects breaking news candi- 
dates according to well-defined breaking news criteria, whose ex- 
act parameters were determined empirically and the correspond- 
ing dataset made available publicly. We have shown how cross- 
language full-text social network searches are used as plausibility 
checks to avoid false-positive alerts. In a first step towards a full 
evaluation, we have shown promising preliminary results and ac- 
tionable next steps in future work for improving the application. 
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